{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "lesbian-springer",
   "metadata": {},
   "source": [
    "## Kubernetes PVC Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a7c5f485",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ[\"NAMESPACE\"] = \"seldon-mesh\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "bf5844d7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'172.19.255.1'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MESH_IP=!kubectl get svc seldon-mesh -n ${NAMESPACE} -o jsonpath='{.status.loadBalancer.ingress[0].ip}'\n",
    "MESH_IP=MESH_IP[0]\n",
    "import os\n",
    "os.environ['MESH_IP'] = MESH_IP\n",
    "MESH_IP"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "758a23f2",
   "metadata": {},
   "source": [
    "### Kind cluster setup\n",
    "\n",
    "To run this example in Kind we need to start Kind with access to a local folder where are models are location. In this example we will use a folder in `/tmp` and associate that with a path in the container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "5a50335d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apiVersion: kind.x-k8s.io/v1alpha4\r\n",
      "kind: Cluster\r\n",
      "nodes:\r\n",
      "- role: control-plane\r\n",
      "  extraMounts:\r\n",
      "    - hostPath: /tmp/models\r\n",
      "      containerPath: /models\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!cat kind-config.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6a4b1d6",
   "metadata": {},
   "source": [
    "To start a Kind cluster with these settings using our ansible script you can run from the project root folder\n",
    "\n",
    "```\n",
    "ansible-playbook ansible/playbooks/kind-cluster.yaml -e kind_config_file=${PWD}/samples/examples/local-pvc/kind-config.yaml\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32da65b2",
   "metadata": {},
   "source": [
    "[**Now you should finish the Seldon install following the docs.**](https://docs.seldon.ai/seldon-core-2/installation/production-environment)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f10b1f79",
   "metadata": {},
   "source": [
    "Create the local folder we will use for our models and copy an example iris sklearn model to it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3391b0fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir -p /tmp/models\n",
    "!gsutil cp -r gs://seldon-models/mlserver/iris /tmp/models"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc04be32",
   "metadata": {},
   "source": [
    "### Create Server with PVC\n",
    "\n",
    "Here we create a storage class and associated persistent colume referencing the `/models` folder where our models are stored."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "e83fafc9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apiVersion: storage.k8s.io/v1\r\n",
      "kind: StorageClass\r\n",
      "metadata:\r\n",
      "  name: local-path-immediate\r\n",
      "provisioner: rancher.io/local-path\r\n",
      "reclaimPolicy: Delete\r\n",
      "mountOptions:\r\n",
      "  - debug\r\n",
      "volumeBindingMode: Immediate\r\n",
      "---\r\n",
      "kind: PersistentVolume\r\n",
      "apiVersion: v1\r\n",
      "metadata:\r\n",
      "  name: ml-models-pv\r\n",
      "  namespace: seldon-mesh\r\n",
      "  labels:\r\n",
      "    type: local\r\n",
      "spec:\r\n",
      "  storageClassName: local-path-immediate\r\n",
      "  capacity:\r\n",
      "    storage: 1Gi\r\n",
      "  accessModes:\r\n",
      "    - ReadWriteOnce\r\n",
      "  hostPath:\r\n",
      "    path: \"/models\"\r\n",
      "---\r\n",
      "kind: PersistentVolumeClaim\r\n",
      "apiVersion: v1\r\n",
      "metadata:\r\n",
      "  name: ml-models-pvc\r\n",
      "  namespace: seldon-mesh\r\n",
      "spec:\r\n",
      "  storageClassName: local-path-immediate\r\n",
      "  accessModes:\r\n",
      "    - ReadWriteOnce\r\n",
      "  resources:\r\n",
      "    requests:\r\n",
      "      storage: 1Gi\r\n",
      "  selector:\r\n",
      "    matchLabels:\r\n",
      "      type: local\r\n"
     ]
    }
   ],
   "source": [
    "!cat pvc.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8384017e",
   "metadata": {},
   "source": [
    "Now we create a new Server based on the provided MLServer configuration but extend it with our PVC by adding this to the rclone container which will allow rclone to move models from this PVC onto the server. \n",
    "\n",
    "We also add a new capability `pvc` to allow us to schedule models to this server that has the PVC."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "fdbebb30",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apiVersion: mlops.seldon.io/v1alpha1\r\n",
      "kind: Server\r\n",
      "metadata:\r\n",
      "  name: mlserver-pvc\r\n",
      "spec:\r\n",
      "  serverConfig: mlserver\r\n",
      "  extraCapabilities:\r\n",
      "  - \"pvc\"  \r\n",
      "  podSpec:\r\n",
      "    volumes:\r\n",
      "    - name: models-pvc\r\n",
      "      persistentVolumeClaim:\r\n",
      "        claimName: ml-models-pvc\r\n",
      "    containers:\r\n",
      "    - name: rclone\r\n",
      "      volumeMounts:\r\n",
      "      - name: models-pvc\r\n",
      "        mountPath: /var/models\r\n"
     ]
    }
   ],
   "source": [
    "!cat server.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b1925df",
   "metadata": {},
   "source": [
    "### SKLearn Model\n",
    "\n",
    "We use a simple sklearn iris classification model with the added `pvc` requirement so our MLServer with the PVC will be targeted during scheduling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8b4be911",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apiVersion: mlops.seldon.io/v1alpha1\r\n",
      "kind: Model\r\n",
      "metadata:\r\n",
      "  name: iris\r\n",
      "spec:\r\n",
      "  storageUri: \"/var/models/iris\"\r\n",
      "  requirements:\r\n",
      "  - sklearn\r\n",
      "  - pvc\r\n"
     ]
    }
   ],
   "source": [
    "!cat ./iris.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "checked-cream",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "model.mlops.seldon.io/iris created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create -f iris.yaml -n ${NAMESPACE}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ab4e3361",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "model.mlops.seldon.io/iris condition met\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl wait --for condition=ready --timeout=300s model --all -n ${NAMESPACE}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ebaabe02",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\r\n",
      "  \"conditions\": [\r\n",
      "    {\r\n",
      "      \"lastTransitionTime\": \"2022-12-24T11:04:37Z\",\r\n",
      "      \"status\": \"True\",\r\n",
      "      \"type\": \"ModelReady\"\r\n",
      "    },\r\n",
      "    {\r\n",
      "      \"lastTransitionTime\": \"2022-12-24T11:04:37Z\",\r\n",
      "      \"status\": \"True\",\r\n",
      "      \"type\": \"Ready\"\r\n",
      "    }\r\n",
      "  ],\r\n",
      "  \"replicas\": 1\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl get model iris -n ${NAMESPACE} -o jsonpath='{.status}' | jq -M ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "upper-cholesterol",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\r\n",
      "\t\"model_name\": \"iris_1\",\r\n",
      "\t\"model_version\": \"1\",\r\n",
      "\t\"id\": \"dc032bcc-3f4e-4395-a2e4-7c1e3ef56e9e\",\r\n",
      "\t\"parameters\": {\r\n",
      "\t\t\"content_type\": null,\r\n",
      "\t\t\"headers\": null\r\n",
      "\t},\r\n",
      "\t\"outputs\": [\r\n",
      "\t\t{\r\n",
      "\t\t\t\"name\": \"predict\",\r\n",
      "\t\t\t\"shape\": [\r\n",
      "\t\t\t\t1,\r\n",
      "\t\t\t\t1\r\n",
      "\t\t\t],\r\n",
      "\t\t\t\"datatype\": \"INT64\",\r\n",
      "\t\t\t\"parameters\": null,\r\n",
      "\t\t\t\"data\": [\r\n",
      "\t\t\t\t2\r\n",
      "\t\t\t]\r\n",
      "\t\t}\r\n",
      "\t]\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!seldon model infer iris --inference-host ${MESH_IP}:80 \\\n",
    "  '{\"inputs\": [{\"name\": \"predict\", \"shape\": [1, 4], \"datatype\": \"FP32\", \"data\": [[1, 2, 3, 4]]}]}' "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58057385",
   "metadata": {},
   "source": [
    "Do a gRPC inference call"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "right-talent",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\r\n",
      "  \"modelName\": \"iris_1\",\r\n",
      "  \"modelVersion\": \"1\",\r\n",
      "  \"outputs\": [\r\n",
      "    {\r\n",
      "      \"name\": \"predict\",\r\n",
      "      \"datatype\": \"INT64\",\r\n",
      "      \"shape\": [\r\n",
      "        \"1\",\r\n",
      "        \"1\"\r\n",
      "      ],\r\n",
      "      \"contents\": {\r\n",
      "        \"int64Contents\": [\r\n",
      "          \"2\"\r\n",
      "        ]\r\n",
      "      }\r\n",
      "    }\r\n",
      "  ]\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!seldon model infer iris --inference-mode grpc --inference-host ${MESH_IP}:80 \\\n",
    "   '{\"model_name\":\"iris\",\"inputs\":[{\"name\":\"input\",\"contents\":{\"fp32_contents\":[1,2,3,4]},\"datatype\":\"FP32\",\"shape\":[1,4]}]}' | jq -M ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "3720658f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "model.mlops.seldon.io \"iris\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ./iris.yaml -n ${NAMESPACE}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b1a93525",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
